ENM Sim Results
Dan Warren
9 May 2016
Simulation results for a study of the relationship between predicting species distributions and the suitability of habitat.
Loading in data…some sims are skipped because they didn’t produce viable distributions to sample from. These will be rerun eventually.
## [1] "./bias_0_rep_10"
## [1] "./bias_0_rep_11"
## [1] "./bias_0_rep_12"
## [1] "./bias_0_rep_13"
## [1] "./bias_0_rep_14"
## [1] "./bias_0_rep_15"
## [1] "./bias_0_rep_16"
## [1] "./bias_0_rep_17"
## [1] "./bias_0_rep_18"
## [1] "./bias_0_rep_19"
## [1] "./bias_0_rep_2"
## [1] "./bias_0_rep_20"
## [1] "./bias_0_rep_3"
## [1] "./bias_0_rep_4"
## [1] "./bias_0_rep_5"
## [1] "./bias_0_rep_6"
## [1] "./bias_0_rep_7"
## [1] "./bias_0_rep_8"
## [1] "./bias_0_rep_9"
## [1] "./bias_0.1_rep_1"
## [1] "./bias_0.1_rep_10"
## [1] "./bias_0.1_rep_11"
## [1] "./bias_0.1_rep_12"
## [1] "./bias_0.1_rep_13"
## [1] "./bias_0.1_rep_14"
## [1] "./bias_0.1_rep_15"
## [1] "./bias_0.1_rep_16"
## [1] "./bias_0.1_rep_17"
## [1] "./bias_0.1_rep_18"
## [1] "./bias_0.1_rep_19"
## [1] "./bias_0.1_rep_2"
## [1] "./bias_0.1_rep_20"
## [1] "./bias_0.1_rep_3"
## [1] "./bias_0.1_rep_4"
## [1] "./bias_0.1_rep_5"
## [1] "./bias_0.1_rep_6"
## [1] "./bias_0.1_rep_7"
## [1] "./bias_0.1_rep_8"
## [1] "./bias_0.1_rep_9"
## [1] "./bias_0.2_rep_1"
## [1] "./bias_0.2_rep_10"
## [1] "./bias_0.2_rep_11"
## [1] "./bias_0.2_rep_12"
## [1] "./bias_0.2_rep_13"
## [1] "./bias_0.2_rep_14"
## [1] "./bias_0.2_rep_15"
## [1] "./bias_0.2_rep_16"
## [1] "./bias_0.2_rep_17"
## [1] "./bias_0.2_rep_18"
## [1] "./bias_0.2_rep_19"
## [1] "./bias_0.2_rep_2"
## [1] "./bias_0.2_rep_20"
## [1] "./bias_0.2_rep_3"
## [1] "./bias_0.2_rep_4"
## [1] "./bias_0.2_rep_5"
## [1] "./bias_0.2_rep_6"
## [1] "./bias_0.2_rep_7"
## [1] "./bias_0.2_rep_8"
## [1] "./bias_0.2_rep_9"
## [1] "./bias_0.3_rep_1"
## [1] "./bias_0.3_rep_10"
## [1] "./bias_0.3_rep_11"
## [1] "./bias_0.3_rep_12"
## [1] "./bias_0.3_rep_13"
## [1] "./bias_0.3_rep_14"
## [1] "./bias_0.3_rep_15"
## [1] "./bias_0.3_rep_16"
## [1] "./bias_0.3_rep_17"
## [1] "./bias_0.3_rep_18"
## [1] "./bias_0.3_rep_19"
## [1] "./bias_0.3_rep_2"
## [1] "./bias_0.3_rep_20"
## [1] "./bias_0.3_rep_3"
## [1] "./bias_0.3_rep_4"
## [1] "./bias_0.3_rep_5"
## [1] "./bias_0.3_rep_6"
## [1] "./bias_0.3_rep_7"
## [1] "./bias_0.3_rep_8"
## [1] "./bias_0.3_rep_9"
## [1] "./bias_0.4_rep_1"
## [1] "./bias_0.4_rep_10"
## [1] "./bias_0.4_rep_11"
## [1] "./bias_0.4_rep_12"
## [1] "./bias_0.4_rep_13"
## [1] "./bias_0.4_rep_14"
## [1] "./bias_0.4_rep_15"
## [1] "./bias_0.4_rep_16"
## [1] "./bias_0.4_rep_17"
## [1] "./bias_0.4_rep_18"
## [1] "./bias_0.4_rep_19"
## [1] "./bias_0.4_rep_2"
## [1] "./bias_0.4_rep_20"
## [1] "./bias_0.4_rep_3"
## [1] "./bias_0.4_rep_4"
## [1] "./bias_0.4_rep_5"
## [1] "./bias_0.4_rep_6"
## [1] "./bias_0.4_rep_7"
## [1] "./bias_0.4_rep_8"
## [1] "./bias_0.4_rep_9"
## [1] "./bias_0.5_rep_1"
## [1] "./bias_0.5_rep_10"
## [1] "./bias_0.5_rep_11"
## [1] "./bias_0.5_rep_12"
## [1] "./bias_0.5_rep_13"
## [1] "./bias_0.5_rep_14"
## [1] "./bias_0.5_rep_15"
## [1] "./bias_0.5_rep_16"
## [1] "./bias_0.5_rep_17"
## [1] "./bias_0.5_rep_18"
## [1] "./bias_0.5_rep_19"
## [1] "./bias_0.5_rep_2"
## [1] "./bias_0.5_rep_20"
## [1] "./bias_0.5_rep_3"
## [1] "./bias_0.5_rep_4"
## [1] "./bias_0.5_rep_5"
## [1] "./bias_0.5_rep_6"
## [1] "./bias_0.5_rep_7"
## [1] "./bias_0.5_rep_8"
## [1] "./bias_0.5_rep_9"
## [1] "./bias_0.6_rep_1"
## [1] "./bias_0.6_rep_10"
## [1] "./bias_0.6_rep_11"
## [1] "./bias_0.6_rep_12"
## [1] "./bias_0.6_rep_13"
## [1] "./bias_0.6_rep_14"
## [1] "./bias_0.6_rep_15"
## [1] "./bias_0.6_rep_16"
## [1] "./bias_0.6_rep_17"
## [1] "./bias_0.6_rep_18"
## [1] "./bias_0.6_rep_19"
## [1] "./bias_0.6_rep_2"
## [1] "./bias_0.6_rep_20"
## [1] "./bias_0.6_rep_3"
## [1] "./bias_0.6_rep_4"
## [1] "./bias_0.6_rep_5"
## [1] "./bias_0.6_rep_6"
## [1] "./bias_0.6_rep_7"
## [1] "./bias_0.6_rep_8"
## [1] "./bias_0.6_rep_9"
## [1] "./bias_0.7_rep_1"
## [1] "./bias_0.7_rep_10"
## [1] "./bias_0.7_rep_11"
## [1] "./bias_0.7_rep_12"
## [1] "./bias_0.7_rep_13"
## [1] "./bias_0.7_rep_14"
## [1] "./bias_0.7_rep_15"
## [1] "./bias_0.7_rep_16"
## [1] "./bias_0.7_rep_17"
## [1] "./bias_0.7_rep_18"
## [1] "./bias_0.7_rep_19"
## [1] "./bias_0.7_rep_2"
## [1] "./bias_0.7_rep_20"
## [1] "./bias_0.7_rep_3"
## [1] "./bias_0.7_rep_4"
## [1] "./bias_0.7_rep_5"
## [1] "./bias_0.7_rep_6"
## [1] "./bias_0.7_rep_7"
## [1] "./bias_0.7_rep_8"
## [1] "./bias_0.7_rep_9"
## [1] "./bias_0.8_rep_1"
## [1] "./bias_0.8_rep_10"
## [1] "./bias_0.8_rep_11"
## [1] "./bias_0.8_rep_12"
## [1] "./bias_0.8_rep_13"
## [1] "./bias_0.8_rep_14"
## [1] "./bias_0.8_rep_15"
## [1] "./bias_0.8_rep_16"
## [1] "./bias_0.8_rep_17"
## [1] "./bias_0.8_rep_18"
## [1] "./bias_0.8_rep_19"
## [1] "./bias_0.8_rep_2"
## [1] "./bias_0.8_rep_20"
## [1] "./bias_0.8_rep_3"
## [1] "./bias_0.8_rep_4"
## [1] "./bias_0.8_rep_5"
## [1] "./bias_0.8_rep_6"
## [1] "./bias_0.8_rep_7"
## [1] "./bias_0.8_rep_8"
## [1] "./bias_0.8_rep_9"
## [1] "./bias_0.9_rep_1"
## [1] "./bias_0.9_rep_10"
## [1] "./bias_0.9_rep_11"
## [1] "./bias_0.9_rep_12"
## [1] "./bias_0.9_rep_13"
## [1] "./bias_0.9_rep_14"
## [1] "./bias_0.9_rep_15"
## [1] "./bias_0.9_rep_16"
## [1] "./bias_0.9_rep_17"
## [1] "./bias_0.9_rep_18"
## [1] "./bias_0.9_rep_19"
## [1] "./bias_0.9_rep_2"
## [1] "./bias_0.9_rep_20"
## [1] "./bias_0.9_rep_3"
## [1] "./bias_0.9_rep_4"
## [1] "./bias_0.9_rep_5"
## [1] "./bias_0.9_rep_6"
## [1] "./bias_0.9_rep_7"
## [1] "./bias_0.9_rep_8"
## [1] "./bias_0.9_rep_9"
## [1] "./bias_1_rep_1"
## [1] "./bias_1_rep_10"
## [1] "./bias_1_rep_11"
## [1] "./bias_1_rep_12"
## [1] "./bias_1_rep_13"
## [1] "./bias_1_rep_14"
## [1] "./bias_1_rep_15"
## [1] "./bias_1_rep_16"
## [1] "./bias_1_rep_17"
## [1] "./bias_1_rep_18"
## [1] "./bias_1_rep_19"
## [1] "./bias_1_rep_2"
## [1] "./bias_1_rep_20"
## [1] "./bias_1_rep_3"
## [1] "./bias_1_rep_4"
## [1] "./bias_1_rep_5"
## [1] "./bias_1_rep_6"
## [1] "./bias_1_rep_7"
## [1] "./bias_1_rep_8"
## [1] "./bias_1_rep_9"Summary statistics
| method | cor.all.spearman | cor.native.spearman | cor.all.pearson | cor.native.pearson | cor.all.hoslem | cor.native.hoslem | train.auc | test.auc | train.max.tss | test.max.tss | train.max.kappa | test.max.kappa |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| bc | 0.3043371 | 0.2355392 | 0.3421751 | 0.2355735 | 170012.45 | 87202.09 | 0.7089074 | 0.6523858 | 0.4391202 | 0.3363781 | 0.1992323 | 0.1916576 |
| brt | 0.1030779 | 0.2530662 | 0.2761195 | 0.2303648 | 127581.66 | 67773.34 | 0.8775116 | 0.7490646 | 0.6350337 | 0.4784065 | 0.4978000 | 0.3186590 |
| dm | 0.5180400 | 0.2098033 | 0.5843531 | 0.2135364 | 80600.59 | 42003.46 | 0.6441629 | 0.6254632 | 0.2775001 | 0.3043340 | 0.1605659 | 0.1793822 |
| gam | 0.3485121 | 0.1714372 | 0.1973356 | 0.1358000 | 193139.56 | 86831.72 | 0.8641932 | 0.7224554 | 0.6162773 | 0.4466814 | 0.4452006 | 0.2887339 |
| glm | 0.1856778 | 0.2588919 | 0.0703818 | 0.2185764 | 244947.76 | 87818.71 | 0.7736123 | 0.7028037 | 0.4637471 | 0.4095257 | 0.3015731 | 0.2664851 |
| mx | 0.3966182 | 0.2703469 | 0.4962945 | 0.2828200 | 84807.10 | 48858.81 | 0.8115005 | 0.7323970 | 0.5181879 | 0.4518322 | 0.3486820 | 0.2923078 |
| rf | -0.1427181 | 0.0626064 | 0.0330461 | 0.0513677 | 157799.02 | 87586.96 | 0.9889538 | 0.7264974 | 0.9590988 | 0.4575120 | 0.8333636 | 0.3864464 |
Relationship between training and test AUC
This plot illustrates the relationship between a model’s ability to predict the data that was used to construct that model vs. its ability to predict a random subset of data that was witheld from the model during fitting. This basically shows what you’d expect, and what you’d hope would be true: that a model that predicts its training data well is generally better at predicting the randomly withheld test data.
##
## Call:
## lm(formula = test.auc ~ train.auc, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.41855 -0.05928 0.01106 0.06975 0.20709
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.23594 0.01544 15.28 <2e-16 ***
## train.auc 0.57479 0.01884 30.50 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.09679 on 1498 degrees of freedom
## Multiple R-squared: 0.3831, Adjusted R-squared: 0.3827
## F-statistic: 930.5 on 1 and 1498 DF, p-value: < 2.2e-16| coef | p | r.sq | |
|---|---|---|---|
| bc | 0.7240088 | 0 | 0.6452058 |
| brt | 0.3878892 | 0 | 0.4123920 |
| dm | 0.6819091 | 0 | 0.6227610 |
| gam | 0.4456828 | 0 | 0.4880493 |
| glm | 0.5110205 | 0 | 0.5946948 |
| mx | 0.4579689 | 0 | 0.6269793 |
| rf | -0.0307403 | 0 | 0.1296183 |
Performance of models
Spearman
Examining performance of models using Spearman correlation coefficient
## [1] "Proportion of models positively correlated with true habitat suitability, native range, Spearman rank correlation: 0.73"## [1] "Proportion of models positively correlated with true habitat suitability, continental scale, Spearman rank correlation: 0.760666666666667"## [1] "Proportion of models positively correlated with true habitat suitability, both native range and continental scale, Spearman rank correlation: 0.609333333333333"
Relationship between Spearman rank correlation in the training region and at the continental scale
This plot and regression demonstrate the relationship between the ability to predict the relative suitability of habitat within the training region and the ability of the model to extrapolate to the continental scale. The clustering of points around the 1:1 line is due to the set of species that occupy all suitable habitat, i.e., the training region and the continental extent are the same. The second plot and regression have those simulations removed.
##
## Call:
## lm(formula = cor.all.spearman ~ cor.native.spearman, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.23903 -0.22049 0.03853 0.24126 0.82966
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.16606 0.01070 15.52 <2e-16 ***
## cor.native.spearman 0.39022 0.02797 13.95 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.3476 on 1498 degrees of freedom
## Multiple R-squared: 0.115, Adjusted R-squared: 0.1144
## F-statistic: 194.6 on 1 and 1498 DF, p-value: < 2.2e-16| coef | p | r.sq | |
|---|---|---|---|
| bc | 0.3915081 | 0.0000000 | 0.3956701 |
| brt | 0.3449451 | 0.0000025 | 0.1102667 |
| dm | 0.2878832 | 0.0000000 | 0.1721463 |
| gam | 0.2599580 | 0.0010083 | 0.0489348 |
| glm | 0.2251982 | 0.0089553 | 0.0312042 |
| mx | 0.3106005 | 0.0000012 | 0.1038302 |
| rf | 0.5131796 | 0.0000000 | 0.2232413 |
##
##
##
## Same plot and regression, with 1:1 simulations removed
##
## Call:
## lm(formula = cor.all.spearman ~ cor.native.spearman, data = new.table[new.table$occupancy <
## 1, ])
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.22015 -0.23283 0.04343 0.25440 0.78304
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.18573 0.01164 15.95 <2e-16 ***
## cor.native.spearman 0.31956 0.03048 10.48 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.3587 on 1353 degrees of freedom
## Multiple R-squared: 0.07513, Adjusted R-squared: 0.07445
## F-statistic: 109.9 on 1 and 1353 DF, p-value: < 2.2e-16| coef | p | r.sq | |
|---|---|---|---|
| bc | 0.3541692 | 0.0000000 | 0.3503571 |
| brt | 0.3003841 | 0.0001225 | 0.0828943 |
| dm | 0.2725324 | 0.0000000 | 0.1595079 |
| gam | 0.1774843 | 0.0350389 | 0.0225855 |
| glm | 0.1557348 | 0.0946511 | 0.0142594 |
| mx | 0.2367083 | 0.0003764 | 0.0629478 |
| rf | 0.5072009 | 0.0000000 | 0.1969817 |
Density plot of Spearman rank correlation between predicted and true habitat suitability, training region only
This plot shows the distribution of Spearman rank correlations between the true relative suitability of habitat and that inferred by each model within the training region. Colors correspond to modeling algorithms.
Density plot of Spearman rank correlation between predicted and true habitat suitability, continental scale
This plot shows the distribution of Spearman rank correlations between the true relative suitability of habitat and that inferred by each model when models are projected to a continental scale. Colors correspond to modeling algorithms.
Density plots of traditional model quality metrics
These plots show the distribution of metrics for AUC, TSS, and kappa on train and test data
Relationships between traditional model quality metrics
These plots show the relationships between AUC, TSS, and kappa on randomly withheld data.
##
## Call:
## lm(formula = test.max.kappa ~ test.auc, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.21239 -0.05653 -0.01180 0.04484 0.38798
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -0.42605 0.01195 -35.66 <2e-16 ***
## test.auc 0.99906 0.01679 59.49 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.08011 on 1498 degrees of freedom
## Multiple R-squared: 0.7026, Adjusted R-squared: 0.7024
## F-statistic: 3539 on 1 and 1498 DF, p-value: < 2.2e-16| coef | p | r.sq | |
|---|---|---|---|
| bc | 0.9371482 | 0 | 0.7987521 |
| brt | 1.0508540 | 0 | 0.7238484 |
| dm | 0.8697910 | 0 | 0.7933622 |
| gam | 0.9869672 | 0 | 0.7195675 |
| glm | 1.0189027 | 0 | 0.7523109 |
| mx | 1.0025411 | 0 | 0.7328574 |
| rf | 0.8084206 | 0 | 0.4679838 |
##
## Call:
## lm(formula = test.max.tss ~ test.auc, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.110990 -0.035348 -0.006096 0.030946 0.176186
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -0.610331 0.007323 -83.34 <2e-16 ***
## test.auc 1.457389 0.010293 141.59 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.0491 on 1498 degrees of freedom
## Multiple R-squared: 0.9305, Adjusted R-squared: 0.9304
## F-statistic: 2.005e+04 on 1 and 1498 DF, p-value: < 2.2e-16| coef | p | r.sq | |
|---|---|---|---|
| bc | 1.472060 | 0 | 0.9383372 |
| brt | 1.546032 | 0 | 0.9202154 |
| dm | 1.283888 | 0 | 0.9278046 |
| gam | 1.525689 | 0 | 0.9275985 |
| glm | 1.484698 | 0 | 0.9319617 |
| mx | 1.563136 | 0 | 0.9466412 |
| rf | 1.377484 | 0 | 0.8845796 |
##
## Call:
## lm(formula = test.max.tss ~ test.max.kappa, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.28713 -0.06988 -0.00405 0.06552 0.37769
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.117450 0.005447 21.56 <2e-16 ***
## test.max.kappa 1.070959 0.017521 61.12 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.09961 on 1498 degrees of freedom
## Multiple R-squared: 0.7138, Adjusted R-squared: 0.7136
## F-statistic: 3736 on 1 and 1498 DF, p-value: < 2.2e-16| coef | p | r.sq | |
|---|---|---|---|
| bc | 1.3029769 | 0 | 0.8083246 |
| brt | 1.0846325 | 0 | 0.6909632 |
| dm | 1.2125668 | 0 | 0.7891727 |
| gam | 1.1546873 | 0 | 0.7192677 |
| glm | 1.1166548 | 0 | 0.7274908 |
| mx | 1.1549751 | 0 | 0.7087974 |
| rf | 0.9297115 | 0 | 0.5627356 |
Relationship between test AUC and Spearman rank correlation, training region only
This plot depicts the relationship between AUC on randomly withheld test data and the ability of the model to estimate the relative suitability of habitat within the training region. The lack of correlation indicates that test AUC is not a good predictor of the model’s ability to estimate the relative suitability of habitat, which is very problematic.
##
## Call:
## lm(formula = cor.native.spearman ~ test.auc, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.09399 -0.22716 0.02415 0.23776 0.73311
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.14800 0.04786 3.092 0.00202 **
## test.auc 0.08569 0.06727 1.274 0.20292
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.3209 on 1498 degrees of freedom
## Multiple R-squared: 0.001082, Adjusted R-squared: 0.0004152
## F-statistic: 1.623 on 1 and 1498 DF, p-value: 0.2029| coef | p | r.sq | |
|---|---|---|---|
| bc | -0.5233939 | 0.0011834 | 0.0476262 |
| brt | 0.6659859 | 0.0015746 | 0.0513475 |
| dm | -0.8518564 | 0.0000163 | 0.0826125 |
| gam | 0.0266830 | 0.8767671 | 0.0001116 |
| glm | 0.6220245 | 0.0003082 | 0.0586268 |
| mx | 0.7106432 | 0.0003714 | 0.0571007 |
| rf | 0.6284545 | 0.0002650 | 0.0598629 |
Relationship between test AUC and Spearman rank correlation, continental scale
This plot depicts the relationship between AUC on randomly withheld test data and the ability of the model to estimate the relative suitability of habitat at a continental scale, where model transferability is an issue. The lack of correlation indicates that test AUC is not a good predictor of model transferability. In fact the (not statistically significant) effect of test AUC on model accuracy is in fact negative.
##
## Call:
## lm(formula = cor.all.spearman ~ test.auc, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.10221 -0.23524 0.04887 0.27383 0.70545
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.37516 0.05501 6.819 1.32e-11 ***
## test.auc -0.18253 0.07732 -2.361 0.0184 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.3688 on 1498 degrees of freedom
## Multiple R-squared: 0.003706, Adjusted R-squared: 0.003041
## F-statistic: 5.573 on 1 and 1498 DF, p-value: 0.01837| coef | p | r.sq | |
|---|---|---|---|
| bc | -0.4846691 | 0.0000010 | 0.1054220 |
| brt | 0.4790017 | 0.0297610 | 0.0246155 |
| dm | 0.0068238 | 0.9611482 | 0.0000110 |
| gam | 0.5456748 | 0.0064947 | 0.0337899 |
| glm | -0.2424256 | 0.2765367 | 0.0054793 |
| mx | 0.6342492 | 0.0010060 | 0.0489528 |
| rf | 0.2782864 | 0.1421057 | 0.0099502 |
Relationship between test max TSS and Spearman rank correlation
These plots and models are similar to those for AUC above, but are instead calculated using the maximum value of the True Skill Statistic. The models are NOT thresholded for this comparision (which would typically be done at the threshold value corresponding to the max value of TSS), rather the max value is used as a quality indicator for the continuous model.
##
## Call:
## lm(formula = cor.native.spearman ~ test.max.tss, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.10708 -0.22742 0.02544 0.24000 0.72415
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.19969 0.02010 9.937 <2e-16 ***
## test.max.tss 0.02034 0.04455 0.457 0.648
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.321 on 1498 degrees of freedom
## Multiple R-squared: 0.0001392, Adjusted R-squared: -0.0005283
## F-statistic: 0.2085 on 1 and 1498 DF, p-value: 0.648| coef | p | r.sq | |
|---|---|---|---|
| bc | -0.3127249 | 0.0033025 | 0.0392650 |
| brt | 0.4213802 | 0.0012613 | 0.0533931 |
| dm | -0.7119542 | 0.0000014 | 0.1025214 |
| gam | -0.0757383 | 0.4854089 | 0.0022559 |
| glm | 0.3524387 | 0.0017322 | 0.0445171 |
| mx | 0.3606235 | 0.0038817 | 0.0379537 |
| rf | 0.3514133 | 0.0029618 | 0.0401497 |
##
## Call:
## lm(formula = cor.all.spearman ~ test.max.tss, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.1077 -0.2370 0.0484 0.2736 0.7135
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.30998 0.02306 13.442 < 2e-16 ***
## test.max.tss -0.15265 0.05112 -2.986 0.00287 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.3684 on 1498 degrees of freedom
## Multiple R-squared: 0.005917, Adjusted R-squared: 0.005253
## F-statistic: 8.916 on 1 and 1498 DF, p-value: 0.002873| coef | p | r.sq | |
|---|---|---|---|
| bc | -0.3299040 | 0.0000004 | 0.1127997 |
| brt | 0.2446686 | 0.0741874 | 0.0166816 |
| dm | -0.0644461 | 0.5395610 | 0.0017449 |
| gam | 0.2646883 | 0.0371636 | 0.0199508 |
| glm | -0.1641416 | 0.2571248 | 0.0059413 |
| mx | 0.3480075 | 0.0038405 | 0.0380402 |
| rf | 0.1894939 | 0.1431953 | 0.0098963 |
Relationship between test max kappa and Spearman rank correlation
These plots and models are similar to those for AUC above, but are instead calculated using the maximum value of Cohen’s kappa. The models are NOT thresholded for this comparision (which would typically be done at the threshold value corresponding to the max value of kappa), rather the max value is used as a quality indicator for the continuous model.
##
## Call:
## lm(formula = cor.native.spearman ~ test.max.kappa, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.14297 -0.22408 0.01945 0.23833 0.70961
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.25569 0.01750 14.610 < 2e-16 ***
## test.max.kappa -0.17387 0.05629 -3.089 0.00205 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.32 on 1498 degrees of freedom
## Multiple R-squared: 0.006328, Adjusted R-squared: 0.005665
## F-statistic: 9.54 on 1 and 1498 DF, p-value: 0.002048| coef | p | r.sq | |
|---|---|---|---|
| bc | -0.6010685 | 0.0000860 | 0.0690623 |
| brt | 0.2134270 | 0.2160077 | 0.0080450 |
| dm | -0.9696937 | 0.0000015 | 0.1020799 |
| gam | -0.0529252 | 0.7204116 | 0.0005942 |
| glm | 0.3317303 | 0.0251044 | 0.0230102 |
| mx | 0.3794972 | 0.0273707 | 0.0223326 |
| rf | 0.0089054 | 0.9520391 | 0.0000168 |
##
## Call:
## lm(formula = cor.all.spearman ~ test.max.kappa, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.14732 -0.23620 0.04346 0.26088 0.75231
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.38970 0.01977 19.712 < 2e-16 ***
## test.max.kappa -0.51980 0.06359 -8.174 6.27e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.3615 on 1498 degrees of freedom
## Multiple R-squared: 0.0427, Adjusted R-squared: 0.04206
## F-statistic: 66.82 on 1 and 1498 DF, p-value: 6.275e-16| coef | p | r.sq | |
|---|---|---|---|
| bc | -0.5387415 | 0.0000000 | 0.1432207 |
| brt | 0.1485466 | 0.4076529 | 0.0036116 |
| dm | -0.1764434 | 0.2178942 | 0.0070202 |
| gam | 0.3320503 | 0.0550221 | 0.0169380 |
| glm | -0.3257679 | 0.0852079 | 0.0136537 |
| mx | 0.2979042 | 0.0729330 | 0.0148114 |
| rf | 0.1066456 | 0.5070174 | 0.0020407 |
Relationship between bias strength and Spearman rank correlation, training region only
This plot and model examine the relationship between the strength of spatial sampling bias and the ability to infer the relative suitability of habitat within the training region.
##
## Call:
## lm(formula = cor.native.spearman ~ bias.strength, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.1238 -0.2253 0.0204 0.2353 0.7361
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.22197 0.01550 14.317 <2e-16 ***
## bias.strength -0.02769 0.02607 -1.062 0.288
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.3209 on 1498 degrees of freedom
## Multiple R-squared: 0.0007527, Adjusted R-squared: 8.567e-05
## F-statistic: 1.128 on 1 and 1498 DF, p-value: 0.2883| coef | p | r.sq | |
|---|---|---|---|
| bc | -0.0012466 | 0.9839170 | 0.0000019 |
| brt | -0.0800899 | 0.2562990 | 0.0067771 |
| dm | 0.0634523 | 0.4283876 | 0.0029062 |
| gam | -0.0748017 | 0.2342651 | 0.0065436 |
| glm | -0.0524343 | 0.4120192 | 0.0031179 |
| mx | -0.0448274 | 0.5357327 | 0.0017780 |
| rf | -0.0147171 | 0.8080144 | 0.0002739 |
Relationship between proportion of suitable habitat that falls within species range and Spearman rank correlation, training region only
This plot and model examine the relationship between the proportion of suitable habitat continent-wide that is within the species range and the ability to infer the relative suitability of habitat within the training region.
##
## Call:
## lm(formula = cor.native.spearman ~ occupancy, data = new.table[new.table$method !=
## "rf", ])
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.14197 -0.22797 0.02571 0.24212 0.68899
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.28506 0.02459 11.592 <2e-16 ***
## occupancy -0.09472 0.04150 -2.282 0.0226 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.32 on 1280 degrees of freedom
## Multiple R-squared: 0.004053, Adjusted R-squared: 0.003275
## F-statistic: 5.209 on 1 and 1280 DF, p-value: 0.02263| coef | p | r.sq | |
|---|---|---|---|
| bc | 0.0075895 | 0.9340359 | 0.0000318 |
| brt | -0.1337044 | 0.1874144 | 0.0091286 |
| dm | -0.0016163 | 0.9891524 | 0.0000009 |
| gam | -0.0287642 | 0.7580792 | 0.0004401 |
| glm | -0.2119707 | 0.0245763 | 0.0231771 |
| mx | -0.2025782 | 0.0581649 | 0.0165161 |
| rf | -0.3449580 | 0.0000926 | 0.0684575 |
Rank correlation vs. bias strength and proportion of suitable habitat within range, training region only
Combining the two above into a single 3D plot and joint model with interactions.
##
## Call:
## lm(formula = cor.native.spearman ~ bias.strength * occupancy,
## data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.13635 -0.22568 0.01772 0.23137 0.76126
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.39617 0.04225 9.377 < 2e-16 ***
## bias.strength -0.23139 0.07109 -3.255 0.00116 **
## occupancy -0.31778 0.07187 -4.422 1.05e-05 ***
## bias.strength:occupancy 0.37333 0.12133 3.077 0.00213 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.3189 on 1496 degrees of freedom
## Multiple R-squared: 0.01466, Adjusted R-squared: 0.01269
## F-statistic: 7.42 on 3 and 1496 DF, p-value: 6.21e-05
Relationship between bias strength and Spearman rank correlation, continental scale
This plot and model examine the relationship between the strength of spatial sampling bias and the ability to infer the relative suitability of habitat when models are transferred to the continental scale.
##
## Call:
## lm(formula = cor.all.spearman ~ bias.strength, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.13096 -0.23261 0.04509 0.27682 0.70467
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.27002 0.01784 15.14 <2e-16 ***
## bias.strength -0.04530 0.02999 -1.51 0.131
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.3692 on 1498 degrees of freedom
## Multiple R-squared: 0.00152, Adjusted R-squared: 0.0008539
## F-statistic: 2.281 on 1 and 1498 DF, p-value: 0.1312| coef | p | r.sq | |
|---|---|---|---|
| bc | 0.0174500 | 0.6502194 | 0.0009537 |
| brt | -0.1124109 | 0.1245471 | 0.0123723 |
| dm | 0.0587055 | 0.2906952 | 0.0051671 |
| gam | -0.0377946 | 0.6095438 | 0.0012097 |
| glm | -0.1270324 | 0.1182486 | 0.0112603 |
| mx | -0.0678401 | 0.3305991 | 0.0043827 |
| rf | -0.0388487 | 0.5546889 | 0.0016181 |
Relationship between proportion of suitable habitat that falls within species range and Spearman rank correlation, continental scale
This plot and model examine the relationship between the proportion of suitable habitat continent-wide that is within the species range and the ability to infer the relative suitability of habitat when models are transferred to the continental scale.
##
## Call:
## lm(formula = cor.all.spearman ~ occupancy, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.12928 -0.23141 0.04823 0.27467 0.68277
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.27481 0.02625 10.468 <2e-16 ***
## occupancy -0.04994 0.04431 -1.127 0.26
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.3694 on 1498 degrees of freedom
## Multiple R-squared: 0.0008471, Adjusted R-squared: 0.0001801
## F-statistic: 1.27 on 1 and 1498 DF, p-value: 0.26| coef | p | r.sq | |
|---|---|---|---|
| bc | 0.1025540 | 0.0712786 | 0.0149832 |
| brt | 0.1105772 | 0.2943397 | 0.0057861 |
| dm | 0.0487699 | 0.5542058 | 0.0016220 |
| gam | 0.0291160 | 0.7907826 | 0.0003265 |
| glm | -0.1498566 | 0.2143942 | 0.0071276 |
| mx | -0.1663268 | 0.1070025 | 0.0119830 |
| rf | -0.3108052 | 0.0012609 | 0.0471087 |
Rank correlation vs. bias strength and proportion of suitable habitat within range, continental scale
Combining the two above into a single 3D plot and joint model with interactions.
##
## Call:
## lm(formula = cor.all.spearman ~ bias.strength * occupancy, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.13460 -0.23143 0.04714 0.27514 0.70411
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.33196 0.04892 6.786 1.65e-11 ***
## bias.strength -0.11388 0.08231 -1.384 0.167
## occupancy -0.11294 0.08320 -1.357 0.175
## bias.strength:occupancy 0.12569 0.14046 0.895 0.371
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.3692 on 1496 degrees of freedom
## Multiple R-squared: 0.002901, Adjusted R-squared: 0.000901
## F-statistic: 1.451 on 3 and 1496 DF, p-value: 0.2264
Relationship between Spearman rank correlation and size of the species’ range, training region only
This plot and model examine whether the size of the species’ range affects the ability to infer the relative suitability of habitat within the training region.
##
## Call:
## lm(formula = cor.native.spearman ~ true.breadth, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.92638 -0.20835 0.02498 0.22518 0.86238
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 3.224e-01 1.220e-02 26.42 <2e-16 ***
## true.breadth -1.432e-05 1.164e-06 -12.29 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.306 on 1498 degrees of freedom
## Multiple R-squared: 0.09167, Adjusted R-squared: 0.09106
## F-statistic: 151.2 on 1 and 1498 DF, p-value: < 2.2e-16| coef | p | r.sq | |
|---|---|---|---|
| bc | -7.50e-06 | 0.0083717 | 0.0317456 |
| brt | -2.73e-05 | 0.0000000 | 0.3141001 |
| dm | 4.30e-06 | 0.2459889 | 0.0062262 |
| gam | -1.28e-05 | 0.0000071 | 0.0893131 |
| glm | -1.81e-05 | 0.0000000 | 0.1714303 |
| mx | -2.43e-05 | 0.0000000 | 0.2423078 |
| rf | -1.69e-05 | 0.0000000 | 0.1672930 |
Relationship between Spearman rank correlation and size of the species’ range, continental scale
This plot and model examine whether the size of the species’ range affects the ability to infer the relative suitability of habitat when models are transferred to the continental scale.
##
## Call:
## lm(formula = cor.all.spearman ~ true.breadth, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.15155 -0.22340 0.03713 0.28082 0.71690
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 2.961e-01 1.464e-02 20.223 < 2e-16 ***
## true.breadth -6.116e-06 1.397e-06 -4.377 1.29e-05 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.3672 on 1498 degrees of freedom
## Multiple R-squared: 0.01263, Adjusted R-squared: 0.01197
## F-statistic: 19.16 on 1 and 1498 DF, p-value: 1.286e-05| coef | p | r.sq | |
|---|---|---|---|
| bc | -5.00e-07 | 0.7693467 | 0.0003989 |
| brt | -1.46e-05 | 0.0000476 | 0.0835960 |
| dm | -4.80e-06 | 0.0647243 | 0.0157081 |
| gam | -9.10e-06 | 0.0078895 | 0.0322227 |
| glm | 5.00e-07 | 0.8978642 | 0.0000765 |
| mx | -1.72e-05 | 0.0000000 | 0.1305917 |
| rf | -1.60e-06 | 0.5914071 | 0.0013362 |
Relationship between test AUC and Pearson correlation, training region only
This plot depicts the relationship between AUC on randomly withheld test data and the ability of the model to estimate the relative suitability of habitat within the training region. The lack of correlation indicates that test AUC is not a good predictor of the model’s ability to estimate the relative suitability of habitat, which is very problematic.
##
## Call:
## lm(formula = cor.native.pearson ~ test.auc, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.96945 -0.19778 -0.00791 0.19447 0.69123
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.14432 0.04225 3.416 0.000653 ***
## test.auc 0.07208 0.05938 1.214 0.225044
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.2833 on 1498 degrees of freedom
## Multiple R-squared: 0.0009824, Adjusted R-squared: 0.0003155
## F-statistic: 1.473 on 1 and 1498 DF, p-value: 0.225| coef | p | r.sq | |
|---|---|---|---|
| bc | -0.4496172 | 0.0003547 | 0.0574775 |
| brt | 0.5811577 | 0.0033410 | 0.0444243 |
| dm | -0.9740110 | 0.0000001 | 0.1236222 |
| gam | 0.2266407 | 0.0802197 | 0.0141012 |
| glm | 0.8607120 | 0.0000001 | 0.1277345 |
| mx | 0.5665322 | 0.0023231 | 0.0421256 |
| rf | 0.4820472 | 0.0001364 | 0.0652908 |
Relationship between test AUC and Pearson correlation, continental scale
This plot depicts the relationship between AUC on randomly withheld test data and the ability of the model to estimate the relative suitability of habitat at a continental scale, where model transferability is an issue. The lack of correlation indicates that test AUC is not a good predictor of model transferability. In fact the (not statistically significant) effect of test AUC on model accuracy is in fact negative.
##
## Call:
## lm(formula = cor.all.pearson ~ test.auc, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.08294 -0.23876 0.03881 0.26916 0.68526
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.18543 0.05178 3.581 0.000353 ***
## test.auc 0.14329 0.07277 1.969 0.049143 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.3471 on 1498 degrees of freedom
## Multiple R-squared: 0.002581, Adjusted R-squared: 0.001915
## F-statistic: 3.877 on 1 and 1498 DF, p-value: 0.04914| coef | p | r.sq | |
|---|---|---|---|
| bc | -0.0085402 | 0.9184805 | 0.0000486 |
| brt | 1.0474117 | 0.0000032 | 0.1082652 |
| dm | 0.0633838 | 0.6358505 | 0.0010399 |
| gam | 0.4399592 | 0.0052959 | 0.0354378 |
| glm | 0.1741865 | 0.3406124 | 0.0042052 |
| mx | 0.8299756 | 0.0000081 | 0.0882975 |
| rf | 1.0407301 | 0.0000007 | 0.1084301 |
Relationship between test max TSS and Pearson correlation
These plots and models are similar to those for AUC above, but are instead calculated using the maximum value of the True Skill Statistic. The models are NOT thresholded for this comparision (which would typically be done at the threshold value corresponding to the max value of TSS), rather the max value is used as a quality indicator for the continuous model.
##
## Call:
## lm(formula = cor.native.pearson ~ test.max.tss, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.97760 -0.19729 -0.00948 0.19586 0.68878
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.18353 0.01774 10.348 <2e-16 ***
## test.max.tss 0.02749 0.03932 0.699 0.485
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.2833 on 1498 degrees of freedom
## Multiple R-squared: 0.0003261, Adjusted R-squared: -0.0003412
## F-statistic: 0.4887 on 1 and 1498 DF, p-value: 0.4846| coef | p | r.sq | |
|---|---|---|---|
| bc | -0.2724049 | 0.0010347 | 0.0487230 |
| brt | 0.3681599 | 0.0027219 | 0.0463077 |
| dm | -0.7919089 | 0.0000000 | 0.1451834 |
| gam | 0.1051196 | 0.1994026 | 0.0076124 |
| glm | 0.5233680 | 0.0000004 | 0.1117083 |
| mx | 0.2780794 | 0.0167666 | 0.0261965 |
| rf | 0.2790858 | 0.0012865 | 0.0469444 |
##
## Call:
## lm(formula = cor.all.pearson ~ test.max.tss, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.08369 -0.23715 0.03949 0.26910 0.68713
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.25120 0.02173 11.558 <2e-16 ***
## test.max.tss 0.08429 0.04818 1.749 0.0804 .
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.3472 on 1498 degrees of freedom
## Multiple R-squared: 0.002039, Adjusted R-squared: 0.001373
## F-statistic: 3.061 on 1 and 1498 DF, p-value: 0.08042| coef | p | r.sq | |
|---|---|---|---|
| bc | -0.0013853 | 0.9798725 | 0.0000030 |
| brt | 0.6207496 | 0.0000090 | 0.0987721 |
| dm | 0.0023480 | 0.9813519 | 0.0000025 |
| gam | 0.2893433 | 0.0036456 | 0.0384628 |
| glm | 0.1328092 | 0.2636155 | 0.0057821 |
| mx | 0.4748551 | 0.0000436 | 0.0746016 |
| rf | 0.6537736 | 0.0000052 | 0.0917834 |
Relationship between test max kappa and Pearson correlation
These plots and models are similar to those for AUC above, but are instead calculated using the maximum value of Cohen’s kappa. The models are NOT thresholded for this comparision (which would typically be done at the threshold value corresponding to the max value of kappa), rather the max value is used as a quality indicator for the continuous model.
##
## Call:
## lm(formula = cor.native.pearson ~ test.max.kappa, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.0136 -0.1864 -0.0110 0.1875 0.6778
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.23951 0.01544 15.511 < 2e-16 ***
## test.max.kappa -0.16304 0.04967 -3.282 0.00105 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.2824 on 1498 degrees of freedom
## Multiple R-squared: 0.007141, Adjusted R-squared: 0.006478
## F-statistic: 10.77 on 1 and 1498 DF, p-value: 0.001053| coef | p | r.sq | |
|---|---|---|---|
| bc | -0.5215903 | 0.0000121 | 0.0850502 |
| brt | 0.1560224 | 0.3353999 | 0.0048848 |
| dm | -1.0702771 | 0.0000000 | 0.1423376 |
| gam | 0.1256522 | 0.2601094 | 0.0058675 |
| glm | 0.5350059 | 0.0000967 | 0.0681048 |
| mx | 0.2842799 | 0.0755591 | 0.0145470 |
| rf | 0.0013232 | 0.9902915 | 0.0000007 |
##
## Call:
## lm(formula = cor.all.pearson ~ test.max.kappa, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.04435 -0.23276 0.04383 0.26808 0.69290
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.35174 0.01891 18.603 < 2e-16 ***
## test.max.kappa -0.24047 0.06082 -3.954 8.05e-05 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.3458 on 1498 degrees of freedom
## Multiple R-squared: 0.01033, Adjusted R-squared: 0.009668
## F-statistic: 15.63 on 1 and 1498 DF, p-value: 8.048e-05| coef | p | r.sq | |
|---|---|---|---|
| bc | -0.0801116 | 0.3135064 | 0.0047028 |
| brt | 0.4451316 | 0.0165916 | 0.0298310 |
| dm | -0.0222049 | 0.8713460 | 0.0001217 |
| gam | 0.3041199 | 0.0253858 | 0.0229227 |
| glm | 0.0687192 | 0.6590125 | 0.0009032 |
| mx | 0.3938497 | 0.0146502 | 0.0272687 |
| rf | 0.1765070 | 0.3321054 | 0.0043555 |
Relationship between bias strength and Pearson correlation, training region only
This plot and model examine the relationship between the strength of spatial sampling bias and the ability to infer the relative suitability of habitat within the training region.
##
## Call:
## lm(formula = cor.native.pearson ~ bias.strength, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.9990 -0.2000 -0.0097 0.1912 0.6979
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.21211 0.01368 15.505 <2e-16 ***
## bias.strength -0.03438 0.02300 -1.494 0.135
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.2832 on 1498 degrees of freedom
## Multiple R-squared: 0.001489, Adjusted R-squared: 0.000822
## F-statistic: 2.233 on 1 and 1498 DF, p-value: 0.1353| coef | p | r.sq | |
|---|---|---|---|
| bc | -0.0114325 | 0.8131033 | 0.0002593 |
| brt | -0.0946265 | 0.1524139 | 0.0107487 |
| dm | 0.0327743 | 0.6618098 | 0.0008875 |
| gam | -0.0837514 | 0.0773772 | 0.0143697 |
| glm | -0.0535361 | 0.3715322 | 0.0036986 |
| mx | -0.0253557 | 0.7059572 | 0.0006603 |
| rf | -0.0152734 | 0.7313223 | 0.0005470 |
Relationship between proportion of suitable habitat that falls within species range and Pearson correlation, training region only
This plot and model examine the relationship between the proportion of suitable habitat continent-wide that is within the species range and the ability to infer the relative suitability of habitat within the training region.
##
## Call:
## lm(formula = cor.native.pearson ~ occupancy, data = new.table[new.table$method !=
## "rf", ])
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.01089 -0.19977 0.00358 0.20865 0.67537
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.23432 0.02207 10.615 <2e-16 ***
## occupancy -0.02734 0.03725 -0.734 0.463
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.2873 on 1280 degrees of freedom
## Multiple R-squared: 0.0004206, Adjusted R-squared: -0.0003603
## F-statistic: 0.5387 on 1 and 1280 DF, p-value: 0.4631| coef | p | r.sq | |
|---|---|---|---|
| bc | -0.0227440 | 0.7510674 | 0.0004669 |
| brt | 0.0896229 | 0.3467935 | 0.0046601 |
| dm | -0.1411295 | 0.2032210 | 0.0074849 |
| gam | 0.0579195 | 0.4114168 | 0.0031260 |
| glm | -0.0541089 | 0.5426437 | 0.0017185 |
| mx | -0.0880597 | 0.3764948 | 0.0036227 |
| rf | -0.0185354 | 0.7786934 | 0.0003664 |
correlation vs. bias strength and proportion of suitable habitat within range, training region only
Combining the two above into a single 3D plot and joint model with interactions.
##
## Call:
## lm(formula = cor.native.pearson ~ bias.strength * occupancy,
## data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.00637 -0.18912 -0.01297 0.19248 0.71104
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.32615 0.03739 8.723 < 2e-16 ***
## bias.strength -0.23330 0.06292 -3.708 0.000217 ***
## occupancy -0.20871 0.06360 -3.282 0.001056 **
## bias.strength:occupancy 0.36454 0.10737 3.395 0.000704 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.2822 on 1496 degrees of freedom
## Multiple R-squared: 0.009508, Adjusted R-squared: 0.007522
## F-statistic: 4.787 on 3 and 1496 DF, p-value: 0.002527
Relationship between bias strength and Pearson correlation, continental scale
This plot and model examine the relationship between the strength of spatial sampling bias and the ability to infer the relative suitability of habitat when models are transferred to the continental scale.
##
## Call:
## lm(formula = cor.all.pearson ~ bias.strength, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.02565 -0.23320 0.04329 0.26548 0.66652
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.33504 0.01672 20.035 < 2e-16 ***
## bias.strength -0.09789 0.02812 -3.481 0.000514 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.3462 on 1498 degrees of freedom
## Multiple R-squared: 0.008024, Adjusted R-squared: 0.007362
## F-statistic: 12.12 on 1 and 1498 DF, p-value: 0.0005139| coef | p | r.sq | |
|---|---|---|---|
| bc | -0.0217736 | 0.4903746 | 0.0022050 |
| brt | -0.1420215 | 0.0623347 | 0.0181661 |
| dm | -0.0123854 | 0.8158145 | 0.0002517 |
| gam | -0.1051944 | 0.0700029 | 0.0151186 |
| glm | -0.1401855 | 0.0351386 | 0.0203852 |
| mx | -0.1573131 | 0.0199420 | 0.0248232 |
| rf | -0.1100824 | 0.1386688 | 0.0101232 |
Relationship between proportion of suitable habitat that falls within species range and Pearson correlation, continental scale
This plot and model examine the relationship between the proportion of suitable habitat continent-wide that is within the species range and the ability to infer the relative suitability of habitat when models are transferred to the continental scale.
##
## Call:
## lm(formula = cor.all.pearson ~ occupancy, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.06854 -0.23424 0.04382 0.26361 0.68340
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.25078 0.02469 10.159 <2e-16 ***
## occupancy 0.06351 0.04167 1.524 0.128
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.3473 on 1498 degrees of freedom
## Multiple R-squared: 0.001548, Adjusted R-squared: 0.0008819
## F-statistic: 2.323 on 1 and 1498 DF, p-value: 0.1277| coef | p | r.sq | |
|---|---|---|---|
| bc | 0.1346311 | 0.0036988 | 0.0383451 |
| brt | 0.1904944 | 0.0823843 | 0.0157958 |
| dm | -0.1354503 | 0.0847317 | 0.0136952 |
| gam | 0.2245293 | 0.0088175 | 0.0313287 |
| glm | 0.1476488 | 0.1355217 | 0.0102858 |
| mx | -0.0313368 | 0.7559816 | 0.0004480 |
| rf | -0.0791234 | 0.4737358 | 0.0023788 |
correlation vs. bias strength and proportion of suitable habitat within range, continental scale
Combining the two above into a single 3D plot and joint model with interactions.
##
## Call:
## lm(formula = cor.all.pearson ~ bias.strength * occupancy, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.01252 -0.23014 0.04414 0.26688 0.68483
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.34721 0.04583 7.576 6.22e-14 ***
## bias.strength -0.19213 0.07712 -2.491 0.0128 *
## occupancy -0.02304 0.07796 -0.296 0.7676
## bias.strength:occupancy 0.17268 0.13161 1.312 0.1897
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.3459 on 1496 degrees of freedom
## Multiple R-squared: 0.01071, Adjusted R-squared: 0.008729
## F-statistic: 5.4 on 3 and 1496 DF, p-value: 0.001072
Relationship between Pearson correlation and size of the species’ range, training region only
This plot and model examine whether the size of the species’ range affects the ability to infer the relative suitability of habitat within the training region.
##
## Call:
## lm(formula = cor.native.pearson ~ true.breadth, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.81004 -0.18476 -0.00645 0.18716 0.81975
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 3.083e-01 1.063e-02 29.01 <2e-16 ***
## true.breadth -1.421e-05 1.014e-06 -14.01 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.2665 on 1498 degrees of freedom
## Multiple R-squared: 0.1158, Adjusted R-squared: 0.1152
## F-statistic: 196.2 on 1 and 1498 DF, p-value: < 2.2e-16| coef | p | r.sq | |
|---|---|---|---|
| bc | -5.80e-06 | 0.0096821 | 0.0305780 |
| brt | -3.02e-05 | 0.0000000 | 0.4364260 |
| dm | 1.04e-05 | 0.0026164 | 0.0411582 |
| gam | -1.42e-05 | 0.0000000 | 0.1907516 |
| glm | -2.24e-05 | 0.0000000 | 0.3002031 |
| mx | -2.43e-05 | 0.0000000 | 0.2806976 |
| rf | -1.62e-05 | 0.0000000 | 0.2856447 |
Relationship between Pearson correlation and size of the species’ range, continental scale
This plot and model examine whether the size of the species’ range affects the ability to infer the relative suitability of habitat when models are transferred to the continental scale.
##
## Call:
## lm(formula = cor.all.pearson ~ true.breadth, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.00513 -0.20291 0.03053 0.24568 0.78587
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 4.234e-01 1.305e-02 32.44 <2e-16 ***
## true.breadth -1.722e-05 1.246e-06 -13.83 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.3273 on 1498 degrees of freedom
## Multiple R-squared: 0.1132, Adjusted R-squared: 0.1126
## F-statistic: 191.2 on 1 and 1498 DF, p-value: < 2.2e-16| coef | p | r.sq | |
|---|---|---|---|
| bc | -1.10e-05 | 0.0000000 | 0.2624811 |
| brt | -3.58e-05 | 0.0000000 | 0.4618756 |
| dm | -5.70e-06 | 0.0207683 | 0.0245027 |
| gam | -1.40e-05 | 0.0000001 | 0.1236380 |
| glm | -9.50e-06 | 0.0019041 | 0.0437457 |
| mx | -2.77e-05 | 0.0000000 | 0.3560798 |
| rf | -2.15e-05 | 0.0000000 | 0.1781500 |
Relationship between test AUC and Hosmer-Lemeshow chisq statistic, training region only
This plot depicts the relationship between AUC on randomly withheld test data and the ability of the model to estimate the relative suitability of habitat within the training region. The lack of correlation indicates that test AUC is not a good predictor of the model’s ability to estimate the relative suitability of habitat, which is very problematic.
##
## Call:
## lm(formula = cor.native.hoslem ~ test.auc, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -229813 -65445 -27183 21758 898869
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 383298 18921 20.26 <2e-16 ***
## test.auc -443280 26593 -16.67 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 126800 on 1498 degrees of freedom
## Multiple R-squared: 0.1565, Adjusted R-squared: 0.1559
## F-statistic: 277.8 on 1 and 1498 DF, p-value: < 2.2e-16| coef | p | r.sq | |
|---|---|---|---|
| bc | -555753.6 | 0.00e+00 | 0.1801005 |
| brt | -562572.8 | 0.00e+00 | 0.2078845 |
| dm | -187793.3 | 5.94e-05 | 0.0720774 |
| gam | -724177.7 | 0.00e+00 | 0.2834255 |
| glm | -714352.3 | 0.00e+00 | 0.2703855 |
| mx | -333254.0 | 0.00e+00 | 0.2196371 |
| rf | -609416.2 | 0.00e+00 | 0.1856885 |
Relationship between test AUC and Hosmer-Lemeshow chisq statistic, continental scale
This plot depicts the relationship between AUC on randomly withheld test data and the ability of the model to estimate the relative suitability of habitat at a continental scale, where model transferability is an issue. The lack of correlation indicates that test AUC is not a good predictor of model transferability. In fact the (not statistically significant) effect of test AUC on model accuracy is in fact negative.
##
## Call:
## lm(formula = cor.all.hoslem ~ test.auc, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -349348 -130605 -70046 35265 2159369
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 600271 36408 16.49 <2e-16 ***
## test.auc -640151 51170 -12.51 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 244100 on 1498 degrees of freedom
## Multiple R-squared: 0.09459, Adjusted R-squared: 0.09399
## F-statistic: 156.5 on 1 and 1498 DF, p-value: < 2.2e-16| coef | p | r.sq | |
|---|---|---|---|
| bc | -1022316.5 | 0.0000000 | 0.1842720 |
| brt | -1028457.4 | 0.0000000 | 0.1948595 |
| dm | -388839.4 | 0.0000000 | 0.1369762 |
| gam | -963178.6 | 0.0000002 | 0.1200715 |
| glm | -556069.2 | 0.0022033 | 0.0425569 |
| mx | -543511.9 | 0.0000000 | 0.2359828 |
| rf | -891803.3 | 0.0000000 | 0.1605362 |
Relationship between test max TSS and Hosmer-Lemeshow chisq statistic
These plots and models are similar to those for AUC above, but are instead calculated using the maximum value of the True Skill Statistic. The models are NOT thresholded for this comparision (which would typically be done at the threshold value corresponding to the max value of TSS), rather the max value is used as a quality indicator for the continuous model.
##
## Call:
## lm(formula = cor.native.hoslem ~ test.max.tss, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -172590 -69373 -32072 21435 918721
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 182482 8065 22.63 <2e-16 ***
## test.max.tss -267228 17878 -14.95 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 128800 on 1498 degrees of freedom
## Multiple R-squared: 0.1298, Adjusted R-squared: 0.1292
## F-statistic: 223.4 on 1 and 1498 DF, p-value: < 2.2e-16| coef | p | r.sq | |
|---|---|---|---|
| bc | -343949.4 | 0.0000000 | 0.1593058 |
| brt | -343336.5 | 0.0000000 | 0.2011186 |
| dm | -122497.8 | 0.0005113 | 0.0544870 |
| gam | -423015.7 | 0.0000000 | 0.2426798 |
| glm | -422072.6 | 0.0000000 | 0.2232601 |
| mx | -190652.5 | 0.0000000 | 0.1855444 |
| rf | -304741.0 | 0.0000020 | 0.0995989 |
##
## Call:
## lm(formula = cor.all.hoslem ~ test.max.tss, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -300905 -137759 -72306 32205 2165365
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 313075 15390 20.34 <2e-16 ***
## test.max.tss -392738 34117 -11.51 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 245900 on 1498 degrees of freedom
## Multiple R-squared: 0.08127, Adjusted R-squared: 0.08066
## F-statistic: 132.5 on 1 and 1498 DF, p-value: < 2.2e-16| coef | p | r.sq | |
|---|---|---|---|
| bc | -646036.8 | 0.0000000 | 0.1699399 |
| brt | -614904.3 | 0.0000000 | 0.1809304 |
| dm | -275202.1 | 0.0000001 | 0.1219005 |
| gam | -592133.4 | 0.0000003 | 0.1138773 |
| glm | -316043.2 | 0.0076080 | 0.0325150 |
| mx | -314118.8 | 0.0000000 | 0.2034503 |
| rf | -446162.4 | 0.0000105 | 0.0861900 |
Relationship between test max kappa and Hosmer-Lemeshow chisq statistic
These plots and models are similar to those for AUC above, but are instead calculated using the maximum value of Cohen’s kappa. The models are NOT thresholded for this comparision (which would typically be done at the threshold value corresponding to the max value of kappa), rather the max value is used as a quality indicator for the continuous model.
##
## Call:
## lm(formula = cor.native.hoslem ~ test.max.kappa, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -130398 -68177 -42884 12078 975609
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 131782 7351 17.927 <2e-16 ***
## test.max.kappa -215716 23645 -9.123 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 134400 on 1498 degrees of freedom
## Multiple R-squared: 0.05264, Adjusted R-squared: 0.052
## F-statistic: 83.23 on 1 and 1498 DF, p-value: < 2.2e-16| coef | p | r.sq | |
|---|---|---|---|
| bc | -429438.01 | 0.0000002 | 0.1182377 |
| brt | -321058.76 | 0.0000055 | 0.1032930 |
| dm | -148198.04 | 0.0021375 | 0.0428038 |
| gam | -451882.38 | 0.0000000 | 0.1493941 |
| glm | -421424.33 | 0.0000000 | 0.1298571 |
| mx | -214534.20 | 0.0000001 | 0.1248340 |
| rf | -61657.82 | 0.4491484 | 0.0026545 |
##
## Call:
## lm(formula = cor.all.hoslem ~ test.max.kappa, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -226072 -142042 -96837 30953 2117115
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 229802 13839 16.606 < 2e-16 ***
## test.max.kappa -285062 44513 -6.404 2.02e-10 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 253100 on 1498 degrees of freedom
## Multiple R-squared: 0.02665, Adjusted R-squared: 0.026
## F-statistic: 41.01 on 1 and 1498 DF, p-value: 2.021e-10| coef | p | r.sq | |
|---|---|---|---|
| bc | -831583.746 | 0.0000000 | 0.1340618 |
| brt | -543015.805 | 0.0000515 | 0.0828726 |
| dm | -323652.119 | 0.0000061 | 0.0904940 |
| gam | -591601.771 | 0.0002217 | 0.0613224 |
| glm | -143671.576 | 0.3575461 | 0.0039203 |
| mx | -337743.066 | 0.0000001 | 0.1249741 |
| rf | -9263.028 | 0.9424451 | 0.0000242 |
Relationship between bias strength and Hosmer-Lemeshow chisq statistic, training region only
This plot and model examine the relationship between the strength of spatial sampling bias and the ability to infer the relative suitability of habitat within the training region.
##
## Call:
## lm(formula = cor.native.hoslem ~ bias.strength, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -74735 -69682 -56990 562 1009818
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 74760 6672 11.206 <2e-16 ***
## bias.strength -4166 11219 -0.371 0.71
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 138100 on 1498 degrees of freedom
## Multiple R-squared: 9.206e-05, Adjusted R-squared: -0.0005754
## F-statistic: 0.1379 on 1 and 1498 DF, p-value: 0.7104| coef | p | r.sq | |
|---|---|---|---|
| bc | -8439.211 | 0.8026380 | 0.0002898 |
| brt | 15752.094 | 0.5953402 | 0.0014874 |
| dm | -3512.234 | 0.8527511 | 0.0001599 |
| gam | -7359.283 | 0.8282318 | 0.0002184 |
| glm | -3664.729 | 0.9146825 | 0.0000533 |
| mx | -4219.059 | 0.8074861 | 0.0002755 |
| rf | -15038.462 | 0.6519629 | 0.0009436 |
Relationship between proportion of suitable habitat that falls within species range and Hosmer-Lemeshow chisq statistic, training region only
This plot and model examine the relationship between the proportion of suitable habitat continent-wide that is within the species range and the ability to infer the relative suitability of habitat within the training region.
##
## Call:
## lm(formula = cor.native.hoslem ~ occupancy, data = new.table[new.table$method !=
## "rf", ])
##
## Residuals:
## Min 1Q Median 3Q Max
## -101030 -64282 -48177 7053 982955
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 31830 10292 3.093 0.00203 **
## occupancy 69376 17368 3.995 6.85e-05 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 133900 on 1280 degrees of freedom
## Multiple R-squared: 0.01231, Adjusted R-squared: 0.01154
## F-statistic: 15.96 on 1 and 1280 DF, p-value: 6.85e-05| coef | p | r.sq | |
|---|---|---|---|
| bc | 47316.64 | 0.3441459 | 0.0041442 |
| brt | -16086.92 | 0.7061654 | 0.0007498 |
| dm | 160550.74 | 0.0000000 | 0.1519308 |
| gam | 65448.95 | 0.1922770 | 0.0078579 |
| glm | 66925.53 | 0.1861099 | 0.0080792 |
| mx | 87840.77 | 0.0005220 | 0.0543163 |
| rf | 77193.47 | 0.1174627 | 0.0113083 |
chisq statistic vs. bias strength and proportion of suitable habitat within range, training region only
Combining the two above into a single 3D plot and joint model with interactions.
##
## Call:
## lm(formula = cor.native.hoslem ~ bias.strength * occupancy, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -150366 -66898 -46529 5741 936242
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -27132 18080 -1.501 0.134
## bias.strength 121452 30423 3.992 6.87e-05 ***
## occupancy 185929 30753 6.046 1.87e-09 ***
## bias.strength:occupancy -230224 51918 -4.434 9.91e-06 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 136500 on 1496 degrees of freedom
## Multiple R-squared: 0.02499, Adjusted R-squared: 0.02303
## F-statistic: 12.78 on 3 and 1496 DF, p-value: 3.011e-08
Relationship between bias strength and Hosmer-Lemeshow chisq statistic, continental scale
This plot and model examine the relationship between the strength of spatial sampling bias and the ability to infer the relative suitability of habitat when models are transferred to the continental scale.
##
## Call:
## lm(formula = cor.all.hoslem ~ bias.strength, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -152408 -146005 -116887 26984 2095217
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 150640 12391 12.157 <2e-16 ***
## bias.strength 2070 20837 0.099 0.921
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 256500 on 1498 degrees of freedom
## Multiple R-squared: 6.587e-06, Adjusted R-squared: -0.000661
## F-statistic: 0.009868 on 1 and 1498 DF, p-value: 0.9209| coef | p | r.sq | |
|---|---|---|---|
| bc | -1627.309 | 0.9788592 | 0.0000033 |
| brt | 33936.683 | 0.5444807 | 0.0019364 |
| dm | -3128.317 | 0.9123579 | 0.0000562 |
| gam | -16553.045 | 0.8112388 | 0.0002646 |
| glm | -7719.950 | 0.9084281 | 0.0000614 |
| mx | 20779.668 | 0.4453391 | 0.0026993 |
| rf | -5024.458 | 0.9237366 | 0.0000425 |
Relationship between proportion of suitable habitat that falls within species range and Hosmer-Lemeshow chisq statistic, continental scale
This plot and model examine the relationship between the proportion of suitable habitat continent-wide that is within the species range and the ability to infer the relative suitability of habitat when models are transferred to the continental scale.
##
## Call:
## lm(formula = cor.all.hoslem ~ occupancy, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -188991 -143326 -108166 23138 2061779
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 199672 18184 10.980 < 2e-16 ***
## occupancy -86942 30693 -2.833 0.00468 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 255800 on 1498 degrees of freedom
## Multiple R-squared: 0.005328, Adjusted R-squared: 0.004664
## F-statistic: 8.024 on 1 and 1498 DF, p-value: 0.004678| coef | p | r.sq | |
|---|---|---|---|
| bc | -98876.92 | 0.2768610 | 0.0054719 |
| brt | -135278.23 | 0.0919920 | 0.0148706 |
| dm | 80632.97 | 0.0546697 | 0.0169868 |
| gam | -244757.04 | 0.0165119 | 0.0263179 |
| glm | -177949.76 | 0.0726879 | 0.0148366 |
| mx | 18309.53 | 0.6503011 | 0.0009532 |
| rf | -52241.19 | 0.5018279 | 0.0020909 |
chisq statistic vs. bias strength and proportion of suitable habitat within range, continental scale
Combining the two above into a single 3D plot and joint model with interactions.
##
## Call:
## lm(formula = cor.all.hoslem ~ bias.strength * occupancy, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -233916 -143984 -103141 22504 2099400
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 144142 33868 4.256 2.21e-05 ***
## bias.strength 110750 56990 1.943 0.0522 .
## occupancy 12906 57609 0.224 0.8228
## bias.strength:occupancy -199131 97256 -2.047 0.0408 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 255600 on 1496 degrees of freedom
## Multiple R-squared: 0.008114, Adjusted R-squared: 0.006125
## F-statistic: 4.079 on 3 and 1496 DF, p-value: 0.006752
Relationship between Hosmer-Lemeshow chisq statistic and size of the species’ range, training region only
This plot and model examine whether the size of the species’ range affects the ability to infer the relative suitability of habitat within the training region.
##
## Call:
## lm(formula = cor.native.hoslem ~ true.breadth, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -215254 -40981 -327 16947 837881
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -3.163e+04 4.222e+03 -7.493 1.15e-13 ***
## true.breadth 1.306e+01 4.029e-01 32.417 < 2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 105900 on 1498 degrees of freedom
## Multiple R-squared: 0.4123, Adjusted R-squared: 0.4119
## F-statistic: 1051 on 1 and 1498 DF, p-value: < 2.2e-16| coef | p | r.sq | |
|---|---|---|---|
| bc | 17.057814 | 0.0000000 | 0.5481931 |
| brt | 15.478361 | 0.0000000 | 0.5743689 |
| dm | 2.772286 | 0.0014253 | 0.0461079 |
| gam | 16.697841 | 0.0000000 | 0.5205979 |
| glm | 16.807351 | 0.0000000 | 0.5186342 |
| mx | 7.024738 | 0.0000000 | 0.3535705 |
| rf | 16.244829 | 0.0000000 | 0.5097342 |
Relationship between Hosmer-Lemeshow chisq statistic and size of the species’ range, continental scale
This plot and model examine whether the size of the species’ range affects the ability to infer the relative suitability of habitat when models are transferred to the continental scale.
##
## Call:
## lm(formula = cor.all.hoslem ~ true.breadth, data = new.table)
##
## Residuals:
## Min 1Q Median 3Q Max
## -389345 -82611 -27523 10720 2204686
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -2.819e+04 8.212e+03 -3.433 0.000614 ***
## true.breadth 2.253e+01 7.837e-01 28.743 < 2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 205900 on 1498 degrees of freedom
## Multiple R-squared: 0.3555, Adjusted R-squared: 0.355
## F-statistic: 826.2 on 1 and 1498 DF, p-value: < 2.2e-16| coef | p | r.sq | |
|---|---|---|---|
| bc | 34.49508 | 0 | 0.6778589 |
| brt | 29.74779 | 0 | 0.5950245 |
| dm | 10.42267 | 0 | 0.2888838 |
| gam | 25.07462 | 0 | 0.2811433 |
| glm | 19.48804 | 0 | 0.1811145 |
| mx | 13.18943 | 0 | 0.5034718 |
| rf | 26.81190 | 0 | 0.5605916 |